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Introduction 



In the fall of 1966, the Bureau of Research of the U,S. Office of 
Bducation li^lcaented a program designed to help smaller Institutions of 
hi g h e r e d uc ation participate In educational research and development 
activities. This program, the Consortium Research Development (CORD) 

Program vas predicated on the "conviction that a ^all, low-budget college 
can contribute to educational research as veil as a big, rich college— If It 
hcc ether saal^ t n xtltu t l ens to help. '* These ***** * institutions have more 
than usual needs to participate In such research and developmental activities. 
^ the first -place, their instructional progr am s hai*e the same need, for 
evalua t io n and further development ae do the larger institutions. Secondly, 
damemetraged ca p ab ili ty to conduct. grant-sponsored reeearch le often vlcved 
as positive evidence of' an institution's excellence. Smaller Institutions, 



hovever, are frequently frustrated . since they of ten lack -the necessary 
instructional research and. jdevelopment capabilities that are. characterlstlc 
of their larger counterparts. 

The GOBD Program vas Initiated In the fall- of' 1966 vith-26 -small colleges 
cosllnked. together to form six consortia for 'carrying on instructional 
research. Institutions comprising, the six consortia mere locatad In North 
Carolina, North and. South Zhdeota, Oregon, Tennessee, Hisslaslppl, Virginia, 



New York and. Wisconsin. 



One of the ebieettves of~ 






consortlB'vas to-combine avallsble. talent 



l oca t ad in participating Institutions In such-a vay -that they could aaalit 
each other In-cosmon re&ear^> development, -and -evaluation acp^tlas-thit 



pertain specifically -to' Instructiottal processes on'the caapue. A s e c ond 
obiectlve vas to stimulate these f aoiltles, • vhl<di are- not cbsractaristlcjlly 
Involved In educational r^earch activities, through exposure to educatlcnal 




Innovation and to the problems and technlqiies appropriate for various types 
of educational research, to Initiate research activities. During the first 
year of the program faculty members, representing many subject matter disci- 
plines, vere exposed in various degrees to the world of educational research. 
As a result of this Initial exposure It became clear that a more Intensive 
form of instruction would be necessary If the faculty participants were to 
develop the necessary research competencies to profit optimally from the 
OOSD activity. 

The declsicn was made to coordinate this training activity through the 
provision of intensive short-term summer irolulag in research md devclcpmant 
competencies. Monies were provided by USOE to offer this specialized train- 
ing. It was proposed that this training be conducted during a two week 
period late In the summer so that participating faculty mconbers could ful- 
fill their obligations to summer sessions at their Institutions and also 
avail themself of this Instruction. This project was Initiated and completed 
to develop the Instructional materials to be used In that Institute and In 
succeeding Institutes both for other COED participants and similar faculty 
groups. 

System Specifications 

Because of the nature of the CORD program It was anticipated that there 
would he a vride variance in the abilities and experience of the participating 
faculty members. Thus It was necessary that the instructional system ap- 
propriate for such an audience be designed such that strengths and weaknesses 
of each participant In each of several topic areas be identified. Based 
upon this Information the system was so designed that It would provide 
learners with the necessary background experience and Individualized pro- 
gram required to make the training period one that would lead to optimal 
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acquisition of the basic skills needed to become involved in educational 
research and Innovative development activities « The system was further designed 
to operate with a very small student- teacher ratio and to provide an abundance 
of materials such that each participating faculty member could be provided 
with a program suited to Ms individual needs. 

The probl^ of diversity in the population relates most directly to the 
amount of learning to be accomplished by each individual during the time 
available. Individuals with extensive experience in the physical and life 
sciences might be expected to require very little additional experience in such 
as Research Design and Data Analysis. Those with extensive experience 
in English and the Himoanities might be expected to require very xxrrie 
additional training in the creative writing tasks involved in proposal and 
report writing, stating objectives in behavioral form, and the preparation 
of prototype instructional material. It seemed unlikely, however, that very 
many participants, since they come from a wide diversity of discipline back- 
grounds, would have developed related competencies in all skill areas relevant 
to conducting educational research. This is especially true for topics such 
as Test Construction, Task Analysis, and Prototype Development. It was 
anticipated that all participants would profit from a reorientation of their 
present competencies and experiences to the context of Behavioral Science. 
Objectives of the Instructional System 

The Instructional System developed as a result of this project was 
designed to develop conpetencies in the following areas: 

1. Specifying objectives in behavioral criteria and form. This includes 
the statement of objectives for instructional systems design purposes, the 
modification and refinement in developmental phases, aind their revision and 
simplification for dissemination. 
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2 . Task analysis and instructional specifications. This Includes 
Information on hierarchical task analysis and techniques for specifying the 
instructional requirements of various kinds to achieve task related behaviors. 

3. Prototype Development. This Includes the experimental operations . - 
and procedures, development of external control strategies, instructional 
monitoring procedures, and evaluation- revision cyclical operations, enqployed 
to maxlml 2 e the learning effects of each instructional element. 

4. Measurement. This includes information on the procedures used in 
the selection and develcpiuent of appropriate criterion measures for various 
types of behavioral objactives, and techniques for assessing absolute criterion 
performance vs. relative student performance. 

5. Research design* This Includes information on "within group" variance 
and the identification of learner variables relevant to the learning experi- 
ence? manlpulable and nonmanipulable variables; examples of research models; 
alternative experimental designs; sources of invalidation; and operations 
research methodology. 

6. Data analysis. This Includes selection of appropriate statistical 
techniques for analysis of the evaluation and/or experimental data and the 
use of computers in the process of analysis. 

7. Proposal and report writing* This Includes information regarding 
components of the research proposal; use of operational language; the key role 
of evaluation of the educational product, and/or the experiment itself — 
including limitations, inferences. Implications, and description of new 
questions raised and additional experimentation required. 
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Develop'jient of the Prototype Instructional System 



The general content areas as described cn the preceding pages vere 
corered In the Instructional system which was comprised of nine modules. 
The modules Included In the Instrictional system and the staff member 
from Teaching Research responsible for the development of each module is 
listed in Table 1. 

Table 1 





Instructional Modules 


and 


Authors 




Instructional Module 










Dr. 


Jack V. Edling 


2. 


Specifying . Ob j ectlves 


Dr. 


Casper F. Paulson 


3. 


Test -Construction 


Dr. 


Del Schalock 


4. 


Task Analysis and Instructional 
Specification 


Dr. 


Paul A. Twelker 


5. 


Prototype Development 


Dr. 


Dale 6. Hamreus 


6.. 


Research Design 


Dr. 


John Gordon 


7. 


Data Analysis I 


Dr. 


James H. Bealrd 


8. 


Data Analysis II 


Dr. 


James H. Bealrd 


9. 


Proposal and Report Writing 


Dr. 


Jack Crawford 



The completed Instructional system was designed to Include, the- following 
•elements for each instructional module: (a) a self-instructional unit designed 

-to provide each, learner with Information covering the topic area as defined by 
the- behavioral objectives for that module, (b) a set of related problems de- 
signed to permit the learn e:r to apply the Information in the Self -Instructional 
Unit to typical problem situations, (c) a criterion test covering the behavioral 
objectives included for the module and for a list -of references wiu.ch would 
permit the learner to pursue further in-depth study of the content covered in 
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the instructional module. The instructional system was designed in such a 
way that it could stand on its own as a self-instructional system or could be 
utilized in a group learning situation. The inclusion of criterion tests for 
each instructional module permitted identification oi those learners who 
would require additional assistance or self-study if they were to demonstrate 
achievement of the behaviors covered la the varictis modules. 

Because of the length of the couple te instruction system those materials 
are not included in this report. Materials are available from Teaching Research 
uxider the title National Research Training Institute Manual . The application 
problems which can be ^ed by the Instnicfcr In nn in-clcss situation or by 
the learner in a self-learning situation are available from Teaching Research 
under the title National Research Training Institutes Workbook . The criterion 
exams prepared for each unit appear in Appendix A of this report. 



Trial of the Instructional System 



A total of 61 participants ^vere exposed to the Instructional System. 

Of this group, 52 were college professors selected by various U.S. Office 
cf Educational Regional Research Directors from those Institutions of higher 
learning cooperating In tie C(^ Program. The remaining nine participants 
were U.S. Office of Education observers representing the Regional Research 
Program and the Bureau of Research. The participants represented a diverse 
population of Interests. The disciplinary backgrounds of the participants 
exclusive of the USOE representatives are shown in Table II. 

Table II 

Disciplinary Background of Participating 
College Professors 



{ 



Discipline N 



Education 1^^ 

Higher Education Administration 10 

Biological Science 4 

Mathematics 4 

English 3 

American Studies 2 

History 2 

Natural Science 2 

Psychology 2 

Sociology 2 

Business 1 

Communication 1 

Economi.es 1 

Guidance 1 

2hisic 1 

Religion 1 

-Speech 1 



Of the 52 college professors, only four had never had a course In psycholo- 
gy, Twenty-one of the participants had, at some tine, written and rubmitted 
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for funding at least one research proposal related to the inprovenent of 



instruction. Of this number > 19 had been funded on one such proposal* 

The learner population was divided into tvo sections so that group 
Instruction could be provided to no score than thirty individuals* Assign- 
ment to sections was made on a random basis. All participants were provided 
with a set of the self-instruction materials. Participants were provided 
with an opportunity to utilize the self-instruction materials before the 
topic was presented by the Instructor* 

Each day the learners were exposed to one instructional module in a 
group paced program sequence in the Teaching Research Automated Classroom 
(TRAC). At the conpletion of the group- paced presentation, participants 
were administered a criterion test for that topic. Based on their scores on 
the criterion test participants were scheduled to either a small group seminar 
or an individual instructional experience. Those individuals who demonstrated 
competence on a given topic were given the opportunity to work on a research 
related activity in an area of their own interest. Those who did not 
demonstrate conq>etence on the criterion examination worked directly with a 
staff member in review of the content until such time that the participant 
and the Instructor were satisfied that competence had been achieved. The 
typical daily schedule was as follows: 

8:00 - 9:45 A.M. Group-paced presentation in the TRAC facility. This 

was a carefully prepared pre-tested learning experience 
cn one of the instructional modules requiring active 
response from the participants and providing feedback 
to the presenter on the percentage of students respond- 
ing correctly to the problem situations as well as an 
individual record of the errors made by each partici- 
pant student. 
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9:45 - 10:00 A.M. 
10:00 - 11:00 A.M. 



11:00 - 12:00 Noon 



12:00 - 1:30 P.M. 
1:30 - 4:00 P.M. 



4:00 - 5:00 P.M. 



7:00 - 10:00 P.M. 



Break 

Administration of the criterion test. The scores 
on each criterion test were used to group the students 
hcBogeneously for further study on the contents of 
that instructional nodxile. 

Independent and small group study. Individuals 
ejqperiencing difficulty in common sub-topics worked 
together under the gt*idance of a staff member in an 
individual tutorial or seminar type situation in 
order to gain confidence in the specified behaviors. 
Lunch 

Independent and small group study. This was a 
continuation of the work undertaken in the morning. 

S(»ae Individuals demonstrated sufficient competence 
in the topic to apply the behavior to the development 
of a research proposal and/or set of materials 
relevant to his own area of interest. 

Group meeting (Interaction). This session was devoted 
to dialogue i^erein ideas and topics of the day were 
examined and in which reaction, comments, questions, 
relevant to those topics were raised and discussed 
by the participants. 

independent study. During this period the participants 
studied the self-instructional materials for the 
instructional module scheduled for the next day. 
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Besults 



Three forae of evaltiatlon of the system were employed: 1) a stiq)le 

pre* post'^test design based on scores on the criterion measure to determine 
what learning gains resulted from the system; 2) participants* written 
appraisals of the system on a prepared evaluation form (see Appendix B); 
and 3) participants' summary and evaluative comments obtained during the final 
two hour institute session devoted specifically to that purpose. Each of 
these evaluations will be summarized below. 

Evaluation 1: Pre~ post-test gain scores 

The following design was established for the pre- post-test measure of 
evaluation. Prior to arrival, participants were randomly assigned to either 
one of two sections. Section 1 did not receive a pre-test, experienced the 
Instructional system and was administered a post- test. Section 2 was pre- 
tested, exp erl enced the instructional system, and did not receive the post- 
test. The results of pre- post-test scores on criterion measures for each 
instructional module and total scores of all measures are shown in Table III, 
(see the following page). 

Inspection of Table III reveals that post-test mean scores were higher 
than pre-test means in all cases. Although mean scores were not comparable 
between criterion measures, the smallast relative mean gain occurred in the 
Objective Analysis Section; whereas the greatest relative mean gain resulted 
in the Bata Anal 3 rsis II Section. Tests of significance were not made since 
the study did not Involve hypothesis testing; however, the size of differences 
between for nearly all measures was sufficient to Indicate that changes 

in behavior in the desir‘>o direction did occur and were sufficiently large 
to insure statistical significance. 
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Table III 



Means, Standard Deviations, Ranges, and Total Possible Points of All 
Separate Criterion Scores and Total Tes*- Scores for Pre- and Post-Test 

Measures. 



Criterion 

Measure 




N 


Mean . 


Standard 

Deviation 


Range 


Total 
. Points 
Possible 


B^avioral 


Pre 


24 


31.29 


11.16 


7-47 


60 


Objectives 


Post 


25 


50.88 


6.41 


35-58 




Objective 


Pre 


24 


10.67 


1.86 


7-14 


18 


Analysis 


Post 


25 


13.84 


2.51 


6-17 




Instructional 


Pre 


24 


15.08 


4.53 


10-22 


44 


Development 


Post 


25 


31.12 


6.74 


17-41 




Measurement 


Pre 


24 


12.52 


2.65 


8-16 


44 




Post 


25 


24.32 


4.05 


15-29 




Experimental 


Pre 


22 


1.45 


1.77 


0-6 


7 


Design 


Post 


23 


5.98 


1.25 


2-7 




Data 


Pre 


24 


6.50 


2.52 


1-11 


20 


Analysis I 


Post 


25 


17.48 


2.20 


10-19 




Data 


Pre 


24 


1.58 


2.94 


0-8 


16 


Analysis II 


Post 


24 


12.71 


2,50 


5-15 




Total, all 


Pre 




79.38 


19.05 


41-115 


209 


Measures 


Post 


22 


156.75 


18.53 


104-178 





Evaluation 2; Individual Written Evaluation of the Tra^n-inp System 

The second form of evaliiation required participants to rate each institute 
topic the written portion and the group presentation — on a five point 
scale. Summaries of those ratings are shown in Table 17, (on the following 
page). 

From Table IV it can be seen that participants rated nearly all topics 
and modes of Instruction at about a four point level. One exception was that 
of the written materials for proposal x/rltlng. Ratings for this section 
averaged 4.7 — nearly perfect. The high rating given the proposal writing 
written materials along with other general comments received In the institute 



suggests that participants were partlcxilarly Interested In obtaining these 
materials and felt they were a high priority need in their subsequent 
proposal, writing efforts. 



Table IV 



Participants* Average Ratings On a 5 Point Scale of Each Institute 
Topic: VJritten Materials and Group Presentations 



TOPICS 


WRiTm WiTEBJMS 


^OUP PRESENTATION 


Orientation 


4.1 


4.1 


Objectives 


3.9 


3.9 


Objectives analysis 


3.9 


3.5 


Measurement 


4.1 


4.3 


Instructional Systems 
Development 


4.1 


4.1 


Research Design 


4.1 


4.1 


Data Analysis I 


4.1 


4.3 


Data Analysis II 


4.3 


4.1 


Proposal Writing 


4.7 


4.1 



General reactions to the instructional system were favorable. 
Several typical comments follow: 

'*As future research directors in our own Institutions, we were made 

aware of some of the problems in research.” 

"The system offers a great opportunity to have expert Instruction 
concentrated in depth and supervised practice in applying new skills in each 

area of research training.” 

"Provided us valuable material to bring home.'* 

"Would like to see the tra5.ning extended with more time to assimilate." 
"Would recomnend to other faculty." 

Evaluation 3; Group Oral Evaluations of the Training 

The third form of evaluation consisted of a two hour group meeting with 
paxticipants staff to summarize and evaluate the training. Again, as 
was noted earlier, the most notable outcome from this experience was the wide 
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variance of difference among participants In terms of their opinions about 
the two weeks training just cookie ted. Areas discussed were the written 
materials, class presentations, testing, pre-session activity, small group 
study, topic schedule sequencing, and participant grouping. 

Some participants felt that the various sections of the written materials 
should have more editorial consistency, others argued to keep the Individualiza- 
tion of authors. One suggestion commonly agreed upon was the desire to ha^^e 
a comaon exanqple carried systematically through each section of the \i?rltteii 
materials. No agreement could be reached, however, regarding what type of 
exanq>le should be used or what discipline should be represented. 

Many participants expressed a desire for some form of follow-up for 
advanced training. Suggestions were made for an advanced Institute, a pre- 
session at the National Convention of the American Education Research Associa- 
tion, and short, small group regional training sessions by the Institute staff 
to be coordinated, whenever possible, during any professional trips made to 
different parts of the country. 

General agreement was expressed that more value than just personal skill 
attainment emerged from the experience. Participants Indicated that they 
acquired broader skills with which to help others at their home Institutions 
In the preparation of proposals and the conduct of Instructional research. 

General Conclusions 

On the basis of the evaluations just reviewed and the experience of the 
staff In carrying out the training, the following conclusions have been made. 
Suggested changes. If a similar Institute were to be conducted, are offered. 
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a. Objectives. In general, the evidence supports the conclusion that all 
objectives of the program were achieved. No major changes In objectives are 
recommended. 

b. Content. The content appeared appropriate for the participants in 
attendance; however, additional attention to individual differences in small 
group activities would be desirable for future formal use of the system. Need 
for development of diagnostic tests to be administered after participants 
Cixperience each xaajor topic, was indicated. The assessment of participants* 
l>ehavioral changes resulting from such diagnostic testing would strengthen 
the system of guiding participants into small groups and independent study 
sessions. 

c. Organization. Although participants differed in their opinions about 
length of program, the conclusion drawn is that two weeks w.as about the right 
amount of time to accomplish the objectives of training. Future use of the 
program should be accompanied with more detailed instruction to participants in 
their use of study time. For example, although it was clearly announced at 
the orientation session that all trainees were free to do what they felt was 
most helpful during the half day time periods devoted to small group study, 
few did anything different than attend the small groups to which they were 
assigned. Several participants did express a desire to have devoted this time 
period to other study efforts. It appears necessary to explicitly set forth 
various study alternatives available to trainees and counsel them into which 
alternative they wish to select. 

While these materials were developed for a specific group of learner- 
participants (faculty members in colleges and universities which are involved 
in CORD consortia), the system has been utilized with other audiences on more 
limited bases. The general conclusion is that the materials can be employed 
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with populations beyond the original target audience. 

There are no claims made that this tzalnlng package is in and of itself 
totally sufficient in producing professional researchers. It vill> however, 
assist the naive researcher to gain enough baseline con5>etence to facilitate 
his interaction with professional researchers and the 



concepts they en^loy. 



Behavioral (^jectives Test 



1. One way in a written behavioral objective for teaching may differ 

from a non-behavioral objective is that the behavioral objective always 
specifies : 

A. Teaching methods 

B. Teacher behavior 

C. Length of a teaching unit 

D. Criteria for measurement. 

2. An ’‘objective," as it has been defined for the purposes of writing 
behavioral objectives, denotes: 

A. A goal that a teacher intends students to accomplish 

B. A desired goal for students to -‘ccomplish 

C. A goal for teachers to accomplish in their teaching methods 

D. A goal teachers would like to accomplish with their students 

3. Generally, the most valid indications of student behavior that are 
related to a behavioral objective are those which: 

A. Reflect the objective indirectly 

B. Foster democratic ideals 

C. Allow the student to express himself 

D. Are linked directly with the objective 

4. The following verbs might be used in writing behavioral objectives 
concerning the testing of geography. Which verb would require the least 
clarification in a behavioral objective? 

A. Understand 

B. Draw 

C. Locate 

D . Identify 

5. The following verbs might be used in writing a behavioral objective 
for teaching high school English. Which verb would require the least 
clarification of a behavioral objective? 

A. Write 

B. Appreciate 

C. Illustrate 

D. Summarize 

6. In a behavioral objective, the audience is: 

A. All the students in a particular grade or level 

B. Some of the students in a particular grade or level 

C. A group of students who are expected to reach the criterion in 
the behavioral objective 

D. A group of students vAo behaved as the objective indicates 

7. The "conditions" of a behavioral objective specify 

A. The setting in which the students’ behavior is to occur 

B. The actions which the teacher will observe 

C. The actions of the leader 

D. Criteria for measuring the student behavior 

S. The "behavioral" aspect of a behavioral objective specifies: 

A. Teacher behavior 

B . Pupil behavior 

C. Behavioral conditions 

D. Measurement of behavior 



II. 



From each of the following groups of objectives select the one objective 
which is most nearly stated in behavioral terms. 

9. A. To teach the students how to build a 3 x 5 inch jewel box. . . 

B. The student will learn the principles of constructing small boxes... 

C. Each 10th grade shop student will build a 3 x 5 inch jewel box... 

D. To show 10th grade students the proper way to construct a 
3x5 inch box. . . 

10. A. To remember the names of the ten provinces of Canada in such a 

way as to. . . 

B. To learn and remember the names of the ten provinces of Canada... 

C. To name and label the ten provinces of Canada on a blank map showing 

only... 

D. To appreciate the Importance of the ten provinces of Canada... 

11. A. To learn the names of the different latitudes of... 

B. To write on an outline map the names of the different latitudes of... 

C. To know the names of the different latitudes of... 

D. To remember how to identify the different latitudes of... 

12. A. To teach the fundamentals of diagraming electrical circuits... 

B. To learn the fundamentals of diagraming electrical circuits... 

C. To diagram an electric circuit with all the fundamentals... 

D. To know how to diagram an electrical circuit. 

13. A. To define the terms decagon, geometry, and equilateral... 

learn the terms decagon, geometry and equilateral... 

C. To know the concepts decagon, geometry and equilateral... 

D. To understand the terms decagon, geometry and equilateral... 

14. A. To explore the identification of various types of vegetation... 

B. To name and describe in writing ten types of vegetation... 

C. To learn the names of ten different types of vegetation... 

D. To know the names of ten dif fer^t types of vegetation. . . 

15. A. To point out five essential points on a map.. 

B. To learn about five essential points on a ma ... 

C. To know and understand five essential points on a map... 

D. To appreciate the value of knowing five essential points on a map... 




n 



III. 



From each of tba following groups of behavioral objectives select the one 
that most accurately describes the des~lrc-d behaviors . 



16. A. Locate ten major oceans, bays and straits on an outline map. 

B. Identify ten major oceans, bays and straits on an outline map. 

C. Write the names of ten major oceans, bays and straits on an 
outline map. 

D. Be able to recognize ten major oceans, bays and straits on an 
outline map. 

17. A. To send a four-word message by Morse code with a blink light. 

B. To send a Morse code message. 

C. To send a message with a blink light. 

D. To send a message using a code. 

18. A. Must be able to read Spanish writing. 

B. Must translate Spanish into English verbally. 

C. Must read a Spanish paragraph and translate orally into English. 

D. Must be able to tell the differences between languages. 

19. A. Must write a Campbell style library paper of at least ten pages. 

B. Must show an ability to write a library paper. 

C. Must write a Campbell style library paper and finish it. 

D. Must be able to write a paper of ten pages or more. 

20. A. Must give 4 examples of methods used to teach biology. 

B. Must write examples of 4 basic instructional techniques in biology. 

C. Must demonstrate an ability to teach biology 4 different ways. 

D. Must show 4 examples of how to teach biology. 

21. A. Find the tea largest cities in Canada. 

B. Locate the position of each of the ten largest cities in Canada. 

C. Write a list containing the ten largest cities in Canada in order 
of s?.ze. 

D. Recognize the rank of each of the ten largest cities in Canada. 

22. A. Write on an isothermal map with a red pencil accurately. 

B. Find t.ae three spots on an isothermal map with heaviest rainfall. 

C. Mark with a red pencil the 3 areas on an isothermal map with 

heaviest rainfall. 

D. Locate and recognize areas of heavy rainfall on an isothermal map. 







IV. 



From each of the following groups of statements select the one which most 
clearly specifies an acceptable level of performance. 



23. A. 

B. 

C. 

D. 

24. A. 

B. 

C. 

D. 



25. 



26. 



27. 



28. 



29. 



A. 

B. 

C. 

D. 

A. 

B. 

C. 

D. 
A. 



B. 

C. 

D. 



A. 

B. 

C. 



D. 



A. 

B. 

C. 



To write a topic sentence suitable for three given related sentences. 
To write a good topic sentence without error. 

To write accurately a topic sentence in 3 minutes. 

To write a sentence for any topic. 

To obtain a score of 50Z on a final test for the course. 

Get a score of 50 or more on a 100 item final. 

Score better than at least half the class on the final test in 
this course. 

Must be able to answer correctly at least 50% of the items on a 
100 question true-false test. 

Write the names of the Canadian provinces on an outline map. 

the ten provinces on an outline map provided in class . 

^^^te the names of at least 7 of the 10 Canadian provinces in a 
5 minute period. 

In five minutes write the names of ten provinces on a Canadian map. 



To underline verbs in sentences accurately. 

To locate and underline verbs in sentences correctly. 

To imderline all verbs in 10 sentences in 15 minutes with 2 or 
fewer errors. 

To write all verbs from 10 sentences on a separate sheet of paper. 

By labeling a given outline map of waterways correctly within 
h hour . 

By being able to look at an outline map and locate waterwa 3 rs correctly. 
By placing waterways on an outline map accurately. 

By labeling without error all the waterways on an outline map in 
30 minutes. 

Must compute accurately to 1 decimal place at 3.east 20 of 30 
given division problems. 

Must work out long division problems in such a way as to demonstrate 
ability. 

Must finish accurately an assignment calling for solution of 
long division. 

Must be able to work 20 long division problems in 30 minutes. 

Must be able to keep time to a given record of music. 

Must clap hands in 4/4 rhythm through ten bars of "Ten Little Indians." 
}^st correctly clap in 4/4 rhythm ^ 4 counts in each measure, to 
a recording of "Ten Little Indians." 

Must be able to demonstrate the ability to keep time to a given record. 







D. 



V. 



From each of the following groups of staresnents select those which describe 
a condition under which an objective is to be measured. 

30. A. Ifcist be able to identify cones, cylinders, and prisms. 

B. Given a set of geometric shapes 

C. Within a period of 30 minutes with less than 3 errors 

D. Students in a 10th grade Geometry class 

31. A. Without the aid of references 

B. 33 correct out of a possible 50 

C. 9th grade geography students 

D. Select the proper location of major rivers. 

32. A. Compute the area of a circle. 

B. Without the aid of a slide rule. 

C. Following the proper formulas. 

D. 9th grade algebra students. 

33. A. Given a problem of the following class. 

B. Select the correct answer in 60Z of the class. 

C. Be able to answer correctly 

B. The entire 12 grade calculus group 

34. A. In a period of less than 1 hour 

B* Without the aid of a reference map 

C. Find the location of a major continent 

D. Correctly in 40Z of all cases 

35. A. By arranging parallel lines on a given map 

B. The student will identify and label 

C. Three of the basic map projections 

B. Will spell all three correctly in a period of 5 minutes 

36. A. The student will solve an algebraic equation 

B. Given a linear equation with one unknown 

C. Within a period of 40 minutes 

i). And follow the correct procedures 



71 . 



Each of the following statenents is a part of a behavioral objective. For 
each statement select the answer which best describes what the statement 
refers to in the objective. 

37. ..within a period of 20 minutes... 

A. Audience 

B . B^avior 

C. Condition 

D. Degree 

33. ..the first year college geography class... 

A. Audience 

B . Behavior 

C. Condition 

D. Degree 

39. ..given a set of carpenters tools... 

A. Audience 

B . Behavior 

C. Condition 

D. Degree 

40 . . .without the use of references . . . 

A. Audience 

B . Behavior 

C. Condition 

D. Degree 

41. ..all auto repair men in electrical circuiting will... 

A. Audience 

B . Behavior 

C . Condition 

D. Degree 

42. ..locate and label... 

A. Audie':ice 

B . Behavior 

C. Condition 

D. Degree 

43. ..with a slide projector and slides... 

A. Audience 

B . Behavior 

C . Condition 

D. Degree 

44 . . . identify the areas containing salt , phosphorous and . . * 

A. Audience 

B . Behavior 

C . Condition 

D. Degree 



VII. 



The following behavioral objectives are followed by a list of the four 
basic requirements necessary for a well stated objective. Select that 
requirement which you feel is least adequately met, or has been omitted 
altogether. 



45. The sixth grade social studies student, given a slate outline map, 
will write the names of continents on it. 

A. Audience 

B . Behavior 

C. Condition 

D. Degree 

46. Given roughly circular shapes with various arrows indicating direction 
the student shall select without error in a 5 minute period all those 
whose arrows indicate a clockwise rotation. 

A. Audience 

B . B^avior 

C. Condition 

D. Degree 

47 . The ninth grade social studies student will locate and name at least 
4 of the 5 climatic areas of Canada. 

A. Audience 

B . B^avior 

C. Condition 

D. Degree 



48. By the end of two months they should be able to type 20 words per 
minute for a period of five minutes with leas than three errors. 

A. Audience 

B. B^avior 

C . Condition 

D. Degree 



49. On an outline map provided, ninth grade geography students will identify 
and label the major rivers of the U.S. an.d Canada. 

A. Audience 

B . Behavior 

C . Condition 

D. Degree 



o 







VIII. 



In the xttost of the following statements of behavioral objectives one or more 

parts have been worded badly or left out completely. For each one select 

the part or parts you think are inadequate and mark the appropriate response (s) . 



30. The student should know the names of three mathematicians and contribu 



tion of 


each to geometry 


A. 


Audience 


B. 


Behavior 


C. 


Conditions 


D. 


Degree 


£. 


All of these 


F. 


None of these 



51. Each student will write a topic sentence suitable for three given 
related sentences. 

A. Audience 

B. Behavior 
C> Conditions 

D. Degree 

£. All of these 
F. None of these 

52. The student will Identify at least three key steps in the proof of 
^T!he square root of two is an irrational number." 

A. Audience 

B . Behavior 

C. Conditions 

D. Degree 

E. All of these 

F. None of these 



53. Given a subject which is consistently classified under the same number, 
to tell from the card catalog where to look for books in that subject. 

(Give call number as far as it consistently occurs.) 

A. Audience 

B. Behavior 

C. Conditions 

D. Degree 

E. All of these 

F. None of these 



54. Students show perception of tonal relationships within a scale (major 
or minor) by singing with syllables or numbers, a famij lar song at a tempo 
established by the teacher. 

A. Audience 

B . Behavior 

C. Conditions 

D. Degree 

£. All of these 
F. None of these 
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55. Given twenty notes written on a staff on the board with the bass clef, the 
class oust write down the names of these notes in one minute, as indicated 

by teacher's start and finish signals, based on teacher's stop watch. 

A. Audience 

B . Behavior 

C. Conditions 

D. Degree 

E. All of these 

F. None of these 

56. Given the titles of five books, to locate the names of the authors in the 
card catalog and write down the authors and titles in acceptable biblio- 



graphical form. 


A. 


Audience 


B. 


Behavior 


C. 


Conditions 


D. 


Degree 


E. 


All of these 


F. 


None of these 



57. Each student will give the root meaning of the terms "geometry," 
"quadrilateral," "decagon," "circumference," and "inscribed." 

A. Audience 

B . Behavior 

C . Conditions 

D. Degree 

E. All of these 

F. None of these 

58. The student will identify all basic shapes (cylinder, cone, prism, cube, 
and sphere) used in familiar buildings and structures. 

A. Audience 

B . Behavior 

C. Conditions 

D. Degree 

E. All of these 

F. None of these 

59. Spell correctly the following words after a 20 minute oral study period: 
cat, dog, bull, white, store. This will be a written exam which will last 

5 minutes. 

A. Audience 

B . Behavior 

C. Conditions 

D. Degree 

£. All of these 
F. None of these 

60. At the end of the two week library session, the pupils will locate, by 
the use of the card catalog, five books named by the instructor. A maximum 
time of five minutes will be allowed. 

A. Audience 

B . B^avior 

C. Conditions 

D. Degree 

E. All of these 

F. None of these 
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Objectiv'i Analysis 
Ci:iter?.on Test 



Which of these is a benefit of pre-design in contrast to 
extenporaneous design? 

a) early planning is diminished. 

b) restoration of the proper emphasis of teacher 
functions is facilitated. 

c) student study tasks are more closely monitored. 



A programed instructional booklet exemplifies: 

a) a superficial choice of terminal objectives. 

b) intentional extemporaneous design of the conditions 
of learning c 

c) pre-design of the conditions of learning. 

d) a wide range of enabling objectives. 



Pre-design of instruction is a waste of tine. 



a) agree 

b) neutral 

c) disagree 



Since we do not know enough about the learning process, it is 
impossible to pre-design instruction to make it more effective. 

a) agree 

b) neutral 

c) disagree 



The best use of an instructor’s time is to know his subject 
matter thoroughly rather than to worry about how to teach it. 

a) agree 

b) neutral 

c) disagree 



as 



6. A student is cheated if he goes to a class T7hen instruction 
is not pre-designed. 

a) agree 

b) neutral 

c) disagree 



7* Pre-design is a poor substitute for extenporaneous design, 

a) agree 

b) neutral 

c) disagree 



8, Pre-design will do alot to improve instruction. 

a) agree 

b) neutral 

c) disagree 



9, If I had my way, all instruction would be pre-designed, 

a) agree 

b) neutral 

c) disagree 



10. In many subjects, extemporaneous design is all right, 

a) agree 

b) neutral 

c) disagree 



11, An objective analyses may produce a "good" hierarchy or a "not- 
so-good" hierarchy. In the long run, a "good" hierarchy is 
one that 

a) has many levels correctly positioned, 

b) includes enabling objectives that are easily accom- 
plished by learrers. 

c) help the instructor in determining what to teach, and 
in what sequence to teach it. 



12. Ilhich of these terms cioes r ot describe ’’enablln?. objectives*’? 

a) I5asic factual and conceptual Vno^’ledpe. 

b) Component behavior or shills. 

c) Loafer order competencies. 

d) Format of tbs instruction. 



13. After identifying the enabling objectives the next step is tot 

a) Identify the type of learning involved in each 
objective. 

b) Develop an instructional environment ^^hich will 
transform learners into graduates who can perform 
at the specified level. 

c) Design instructional sequences that reflect the 
basic learning principles. 



14. After establishing the terminal objective (s) the designer 
should ask the following auestion. 

a) IJhat level on the hierarchy of learning is this 
objective? 

b) T.Jhat media will best facilitate reaching this goal 
without losing the individualized approach? 

c) TTiat would the student have to know to perform this 
objective successfully? 

d) I'lhat in the Tzorld do I do nov?? 



15. PelovT is a set of objectives. Tvjo of these are enabling 
objectives. One. is a terminal objective. 

1) Represent forces and their directions as parts 

of triangles. 

2) Name parts of a triangle. 

3) l^ame horizontal and vertical compon^r.ts of forces 
as vectors. VJhich is a terminal objective? 

a) 1 

b) 2 

c) 3 
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16. Below is a set of objectives. One is a terminal objective 
and the others are enabling objectives. 

1) The student will make a left turn with a stan- 
dard shift car onto a highway from a stop. 

2) The student will discriminate the clutch from 
the gas pedal and the brake. 

3) The student will disengage the clutch. 

4) The student will signal for a left turn. Which 
of the following is a terminal objective for this 
set? 

a) 1 

b) 2 

c) 3 

d) 4 



17. In the following objective analysis, choose the correct 
ordering of enabling objectives. 




IVa 



1 . 

2 . 

3. 

4. 

5. 




Distinguish between equal 
and unequal expressions. 
Perform operations of 
multiplication and division 
Recognize false math 
sentences 

Recognize the math sentences 
Recognize true math 
sentence''? 



a. b. c. d. 

Ila - 1 2 5 3 

lib - 4 1 3 5 

III - 2 4 2 4 

IVa - 5 3 4 1 

IVb - 3 5 1 2 
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18. Read the followinp list of objectives for a math problem. 

1) T*ead 1 and 2 place numerals. 

2) Multiply a two place whole ntjinber by a one place 
whole number. 

3) T'^ultiply a one place whole number bv another one 
place whole number. 

4) Add a group of ten (s) to a tvro place i^hole number 
(carrj^ing) . 

T^iich diagram is closest to revealing the hierarchy? (The numbers 
in the squares refer to the numbered objectives above.) 



a) 



I 

T 




3 





b) 



1 



2 




I 

J 

i 



T 




j 



\ 



1 



5 



i 




3 




4 




r 



1 



I 



i 







Prototype Development 
Criterion Test 



For test items 1-8, circle the answer you consider best: 
then push the same letter button on your response station. 

1. The stages defliied in the instructional system develop- 
ment section are: 

A. Identifv behavioral objectives" determine enabling 
objectives, produce the Instructional product; 
analyze tryout results. 

B. Produce Instructional product; conduct tryout of 
product; analyze tryout results; modify product; 
recycle . 

C. Determine enabling objectives; produce instructional 
product; construct performance measures; conduct 

tryout; analyze ree'jlts; recycle. 

D. Produce instructional product; construct performance 

measures; conduct try-out; analyze results; recycle, 

2. The process of systematically following the instructional 
’’blue prints” or specif icatidns is to: 

A. Produce the instructional product 

B. Identify types of learning 

C. Identify behavioral objectives 

D. Conduct try out of the product 

3. To translate the analysis of try-out results i.nto changes in 
the Instructional materials is to; 

A. Determine enabling objectives 

B. Produce the instructional product 

C. Pvecycle the development process 

D. Modify the product 

4. Obtaining data from observations and other evaluations of 
the system such that weaknesses can be identified is to: 

A. Construct performance measures 
Determine enabling objectives 

C. Conduct try-out of the 

D, Analyze trv-out results 
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. The stage of instructional system dcvalopment vliose purpose 
is to determine whether the instructional system achieves 
its objective is the: 

A, Analysis of try-out results 

B, Identiflx.ation of events that provi.de conditions of 
leamine 

C, Deteminaticn of enabling objectives 

D, Production of the instructional product 

6. Recycling is concerned vjitb 

A. Determining enabling objectives for all behavioral 
objectives 

B. Administerine and analyzing performance measures 
ModifyjLng all phases of the instructional product 

D. Reduplicating all instructional system development 
stages 

7. The team of specialists considered essential to produce the 
instructional product includes ar 

A, Content specialist, media specialist, behavioral 
scientist 

B, Production specialist, measurement specialist, field 
monitor 

C, Subject matter specialist, field monitor, liaison 

D, Production specialist, try out specialist, measureirent 
specialist 

8. The continuum of experience model is most usefi«l in 

A, Constructing performance measures 

B, Internreting results of the trv-out 

C, Deciding x^hat media to use 

D, Deciding hox^ to conduct the try-out 

For test items 9-16, circle whether the statement is true 
or false and push the same letter button on your response station. 

9. Try-out of the instructional product occurs after all develop- 
ment has been completed, 

A. True 

B. False 



3 I 

er|c 



10. Try-out of the instrucLionr.l product should only he conducted 
with learners of apnropriate prade and ahllltv level. 

A. True 

B. False 

11. Analysis of try-out results cannot put morn confidence in 
test results than in comments from try-out subjects. 

A. True 

B. False 

12. Fip.h error rates on a criterion test indicate the Instruc- 
ional system has portions that are relatively ineffective. 

A. True 

B. False 

13. Try out following instructional system modification is 
crucial . 

A. True 

B. False 

14. If it is possible, the sane subject should be used to try- 
out all revisions of a particular se 5 »ment of the new in- 
structional system. 

A. True 

B. False 

15. Product development reauires translating in: ructional 
specifications into inrltten statements and/..r media forms. 

A, True 

B. False 

16. Incidental details having little to do with the basic 
content of the new Instructional system ofti^n interfer TdLth 
effective learning during the try-out. 

A. True 

B . False 
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For test if^jns 17-23, circle the ^^nswer you consider best 
when all persons have connleted t'.ese iteris you -^11 be in- 
structed to push the appropriate button on your response station. 

17. .Analysis of try— outs is used to: 

A. Plan modifications to the system 

B. Determine weak teachinp strategies 

C. Determine unrealistic specifications 

D. All of these 

E. ’Tone of these 

18. To Tueaeure the first try-out of a seenent of a new instruc- 
tional system 

A. requires a validated test 

B. Demands small groups 

C. Involves only learners 

D. All of these 

E. Pone of these 

19. Modifications to the new instructional system follo^d-ng 
analysis are planned iTitht 

A-. The same team of specialists used in production 

B. A new team of specialists 

C. The content specialist only 

D. All of these 

E. Done of these 

20. The process of producing the instructional product in- 
volves 

A. Systematically following the instructional specifi- 

cations 

B. Translating specifications into prototype 

C. liaison between a team of specialists 

D. All of these 

E. Done of these 
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21. To modify the instructional materials fcliovTing try-out 
requires 

A. Translating the analysis results into changes 
in the product 

B. Planning Td.th the development specialist team 

C. Revising enabling objectives 

D. All of these 

E. None of these 

22. In teaching learners to discriminate between sounds 
produced from an oboe and an English horn, according to 
the continuum of experience model, use 

A. Direct experience 

B. Objective codification (audio) 

C. Subjective codification (audio) 

D. All of these 

E. None of these 

23. P.esults of an analysis of the total Instructional system 
indicates that learners achieved all the stated enabling 
objectives but did not achieve the terminal objective. 
1‘jhich interpretation would be appropriate 

A. Enabling objectives were irrelevant to the terminal 
objective 

B. Enabling objective stated required learning steps 
that were too big. 

C. Enabling objectives were too easy 

D. All of these 

E. 'Tone of these 
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24. DuritJg the first session in the Institute on Specifying 
Behavioral Objectives vou inrote two objectives in the 
area of your discipline. For this part of tre test, 
copy one of those objectives in the space below and do tba 
following; ^Irst, identify one enabling objective 
necessary to attain the teminal behavior you stated. 

Then: 



A. Describe hor-i you would proceed to produce the in- 
structional segr-ept for that enabling objective. 

B. Detail how you ’rould plan to try-out the segment pro- 
duced. In your discussion give consideration to all 
pertinent aspects of production and try-out covered 
in this section. 
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Figure 10 



Look at Figure 10. The longitude of Point A 
is written (?) • 

The longitude of Point B on Figure 10, is 
writte n- (?) 

The longitude of Point C on Figure 10 is 
written^ 

Longitude is (?) - (?) distance or 
location. 

Longitude lines are numbered from 0 at the 
ceridian to (^) ° 
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8-35 On Figure 11 we have an imaginary map with 
with imaginary cities. See if you can read 
their longitudes. Use Figure 11 for the rest 
of this set. The longitude of Conlin is 

(? ) ° (7) • 

8-36 The longitude of Elten is (?) ^ (?) * (?) » 
8-37 The longitude of Dagman is (?) ^ (?) ' (?) 

8-38 The longitude of Frantin is {21 • 

8-39 The longitude of Bantle is £?) » 
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Error Rates Per Frane by Grade and Ability Level 
555666777 



Frame 

Number 


Low 

N-18 


Ave. 

N-16 


High 

N-18 


Low 

N»9 


Ave. 

N-19 


High 

N*20 


Low 

N-12 


Ave. 

N»14 


High 

N-5 


8-30 


22 


0 


6 


11 


5 


0 


0 


0 


0 


8-31 


6 


6 


6 


11 


0 


0 


0 


0 


0 


8-32 


22 


6 


0 


0 


11 


5 


0 




0 


8-33 


22 


13 


0 


11 


11 


0 


0 


7 


0 


8-34 


17 


19 


6 


11 


11 


0 


8 


0 


0 


8-35 


33 


50 


28 


44 


26 


5 


17 


7 


0 


8-36 


39 


38 


23 


44 


26 


10 


25 


21 


0 


8-37 


61 


31 


17 


44 


32 


10 


17 


0 


0 


8-38 


22 


25 


11 


22 


26 


5 


17 


0 


0 


8-39 


33 


19 


11 


44 


42 


0 


8 


0 


0 



A. Identify those frames judf',ed and for which students 



Bad Frames T-^ich Students 
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Test Over heasurement 



Part I* Foundations 

Items appearing in Part I of the test are designed to assess knowledge 
that is basic to the use o:’ measurement in instructional ^sterns development and 
research. In responding to these items place a ,T in front of the items which 
are true and an F in front of those which are false. 

1, Instructional research and instructional evaluation refer to essen- 
tially the same process. 

2. Demonstrating the effectiveness of an instructional system in one 

school setting does not mean that it will be effective in another. 

3* The aim of instructional evaluation is to determine whether a par- 
ticular instructional system brings about the educational outcomes 
(behavioral objectives) that are intended to derive from it. 

4* Persons interested in instructional systems evaluation must become or 

must rely heavily upon a person competent in measurement in the 
behavioral sciences. 

5* Because of the "inexactness” of measurement in education and the 

behavioral sciences two or more measures should be used to assess 

any characteristic or outcome that is being considered in a particular 

study. 

- 6. The concepts of reliability and validity enter educational measurement 

only because constant error appears in all such measures. 

- 7* The concepts of the relevance, representativeness and fidelity of 

items in a measure have as much to do with the problem of constant 
error as does the concept of ’^construct” validity. 

.. The control of "random” error in measurement is one of the most 

bothersome and difficult problems that educators face in carrying 
out instructional research and evaluation. 

T 
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9* The essential differen *. tvreen *aeasui*ement in the physical sciences 

and the behavioral sciences lies in the fact that physical sciences 
rely upon direct measurement, i#e, measurement of observable proper- 
ties, “while the behavioral scimces rely upon indirect measurement, 
i#e#, measurement of ^indicants*' of inferred properties. 

10. The accuracy of a measure is synonymous vdth its reliability. 

Part II. Applications 

Items appearing in Part II of the test are designed to assess the ability 
to apply basic knowledge about measurement to the solution of tasks demanding of 
that knowledge. In responding to the first item identify the elements (proper- 
ties, traits, characteristics, outcomes) that can be measured only indirectly. 
i.e., that are hvtxtthetical constructs . Do this by placing a check in front of 
these elements. 

Ability to solve story problems in mathematics 

Knowledge of ancient history 

Personality 

^ Intelligence 

_______ Anxiety 

Time required to run the hundred yard dash 

______ Considerateness 

Orientation to the use of questions 

Citizenship 
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In responding to the second iteia in part II of the test indicate the 
level(s) of laeasureiuent (colunin 2) that are aonrorriate and feasible vithnn a 
school setbinf: for the aeasursment of the cognitive outcomes listed in column 
1. Do this hy placing the number descriptive of the level (s) of measurement 
in front of the cognitive outcome that is to be measured. Wiere more than one 
level is appropriate rank them according to the Tx>ver of the data that would 
cone from each, listing the most powerful first. 



Rules governing basketball 

History of the Eskimo 

Sensitivity to the feelings 
of others 

Competence in math 

Cco^tence in marbles 

Use of good table manners 

Reading skill 

Writing skill 

Self confidence 

Tolerance of individual 
differences 



1. Knowledge level 

2. Situational response 

level 1: Described 

criterion behavior 

3* Situational response 
level 2: Related 

criterion behavior 

4# Situational response 
level 32 Isomorphic 
criterion behavior 



er|c 
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Part HI, Instrument Development 

Items appearicg in Part in of the tost are designed to assess knowledge 
that is required in order to develop instruments for use in educational neasure- 
ment. In responding to the first item, match the various attributes of a 
measure that are listed in column 1 with the phrase in column 2 that most appro- 
priately describes the kind of evidence required in support of that attribute. 

Theoretical clarity of items 

Conceptually appropriateness of 
items 



An individual's scores on a 
test taken twice "with two weeks 
intervening 



An individual's scores on 
alternate forms of a test 

The correlation between an 
individual's performance on two 
measures that are known to be 
related 

The extent to which an item 
differentiates between those 
scoring hi^ and low on a test 

The extent to which an item 
calls for behavior that is 
isomorphic to the property that 
is being measured 



Relevance 1 , 

Representativeness 2. 

Fidelity 

3 

Reliability 
Construct validity 

4 * 

5 . 

6 . 
7 . 



In respondiiig to the second itec in this part of the test indicate the 
class(es) of neasureaent (colmsn 2) that are arpropri^te.and feasible with^^ 
school setting for the neasurement of the cognitive outcomes listed in colucn 1. 
Do this hy placing the n'jmber descriptive of the class (es) of measurement in 
front of the cognitive cutcome to be measured* IfJhere more than one class is 
appropriate than according to the p ower of the data that yo^ild come fr o m 

each* listing the iDost powerful first* 



Rules governing basketball 
History of the Eskimo 



1* Interview 
2* Systematic observation 



Sensitivity to the feelings 
of others 



3* Standardized objective measures 
(Paper and Pencil) 



Competence in math 
Competence in marbles 
Use of good table manners 



4* Standardized projective measures 
(Paper and Pencil) 

5* Teacher-made tests (Paper and 
Pencil) 



Reading ddll 
Writing skill 



6. Unobtrusively obtained products 
and/or records 

7* Unobtrusive observation 



Self confidence 




CRITERlCXj TEST: EXTEKTS^TAL DESICI 



Directions: Reed the proposal extract and answer ^e questions 

that follow* 

III, Objectives 

Bie primary objective of the proposed research is to determine 
whether or not student attitudes towards teadiing objectivei* in- 
cluded in teacher education programs are effectively altered after 
they have had eiq)erience in simulated settings, Ihe experitoentation 
will be designed to answer questions x^ich are specific to the 
particular e:q>eriencas included in the materials. The simulation 
problems are> in this case, limited to situations of a personal- 
social nature. 

Specific questions to be answered are the following: 

1. Will the experiences provided through individualized 
instruction in the simulation facility (as described by Xersh, 1963), 
produce changes in attitude in a positive direction toward the 
subject matter of educational psychology? 

2. Will (dianges in attitude become evident after the learner 
has had eiqperience with one program of 20 problem sequences? 

3. Will dianges in student attitude maintain in strength 
following the termination of the learner's experience in the 
simulated setting? 

IV, Procedure 



A, General Hethod and Research Strategy 

1, Sample Plan and Procedure, The experimental subjects 
will be selected from the students a.*: Oregon College of Education 
majoring in elementary education. They will complete their in- 
struction in the classroom simulator within one academic year of 
their practice-teaching experience or internship. The subjects 
^d,ll be screened on the basis of previous educational background 
and eiqperience to insure homogeneity in this regard. Also, they 
will be screened on the basis of their performance on the pretest 
of attitude toward specific teaching objectives selected from those 
included in education courses. In so far as possible, only those 
students \^ose performance on the pretest indicates that they are 
neutral or negative toward the specific teaching objectives will be 
included. Subsequently, the subjects will be assigned to either 



the experimental or the control ;roup randoniy on the o- 

sAolStic aptitude, sex, and other selected variables which other- 
wise laigjit bias the findings. 

0 Treatments. The classroom simulation technique under con- 
siderakon atten?,tr to create for the student teacher all of the 
relevant features of a single classroom situatxon. 

In brief, the instructional procedure x^hich has been developed 
for experimentation with the slmulatron materials is ^ follws: 
First^e student teacher (T) is oriented to the simulation facility 
and to the procedure, then he is given a perfontance t^t 
sliBulated classroom using one of the three programs, ^e 
tion and pretesting procedure takes approximately one and 
hours per siibject. Hext, T is given the actual Instraction in tte 
simulation facility. The filmed problem sequen^s of act^l c^s- 
room Situations are presented and T is requested to enact Ms res 
nonse to each. Depending upon the reaction of T, the experi^ntor 
^E) selects and projects one of two or three alternative feedback 

sequences . 

3. Controls. The control group ^rtll he treated 
resnect the same as the e^qperimental group except the inttvidua*s 
in the control group will not experience instruction in the sx^ 
Sated setting. Differences in attitude between the expen^ntal 
and control groups nay be attributed only to experience in 
simulation facility • 



S. 



Data Types to be Gathered end Methods to be Used. 



As is indicated above, the experimental group will "“^ergo 

In the simulation faciUty which may be conveniently 

experience XU uu ^ , /,x of the pretest period 

SSn^Sud^s Srfirr 2 o'pr;hie;%;que;»s; (2) after the In- 
Sructional period, and, (3) after 

Ssts^ll be administered to individuals in the /ronp 

before their experience in the simulation facility begins. 
eacS ofSS in^ructional phases indicated above. Approxl^tely 
six weeks after termination of the simulation 

m-ftPrlon test will be administered once again. Individuals in 
tS control group, which .ri.ll not haw experience in the simlatxon 
facility, will he tested on a time schedule approximating that of 
Individuals in the experimental group. 
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QUESTIONS: 



What is the experimental unit? •. 

2. What is /are the experimental treati 2 ent(s)? 

3. What is the method of assignment? 

4. Diagram the design using R*s, X*s, and O’s. 

5. List sources of invalidity that are accounted for. 



6. List sources of invalidity that are not accounted for. 



7. Redesign the proposal, give both new diagram and brief 
explanation. 



1 



Data Analysis I 
Criterion Examination 



• i^sume that the nuaibers 6> 13, 24 were taken from a ratio 
scale. Which of the following sets of nimlbers would be 
appropriate substitution? 



a. 13, 20, 31 

b. 24, 52, 96 

c. li, 25, 47 

d. 1, 6, 14 



2. If the numbers 4, 17, 43 were taken from an ordinal scale, 
which of the following sets would be an appropriate substi”- 
tution? 



a. 1, 2, 3 

b. 4, 16, 12 

c. 13, 10, 26 

d. none are appropriate 

. If the numbers 6, 21, 24 represent an interval scale which 
set of numbers would be an appropriate substitution? 



a. 


11, 


40, 


45 


b. 


12, 


42, 


48 


c. 


18, 


63, 


72 


d. 


21, 


35, 


39 



For each of the following kinds of measures Indicate whether the 
scale is nominal, ordinal, interval, and ratio 

4. Human age in years. 

5. IQ 

6. Numbers on pages in a book 

7. Notes on the musical scale 

8. Tenperature on a centigrade scale, 

9. Elapsed time in seconds 

10, Calendar years 
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11. An investigator wished to determi^ie whether physical coordi- 
nation training was effective in iinproving reading 
effectiveness. He selected a sarnie of “below grade place- 
iDent“ readers and gave them 20 hours of training in lateral 
coordination. A second san^le was chosen from the same pool 
of low readers and given no training. For analysis purposes 
he should consider the samples 

a. independent 

b . related 

12. A second investigator stxidying the same problem selected only 
one group of low readers and gave them training. He then 
compared pretraining and post- training reading performance. 
His sauries should be considered 



13. A third investigator thought that differences in training 
effectiveness might be expected for boys and girls. He 
designed his study to have training and no training groups of 
boys and girls. How many samples would be required for 
analysis purposes? 



14. The null hypothesis which states that "the differences 
between means and variances of sanq>les are no greater than 
differences due to the vagaries of random sampling from a 
single, normally distributed, infinite population," is best 
suited for 

a . causal-comparative studies 

b . experimental stxxdies 

c. exploratory studies 

d. studies of physical characteristics only. 

15. Causal-cois 5 >arative studies differ from experimental studies 
in that 

a. Causal-comparative studies look for differences while 
experimental studies only look for change. 

b. Causal-conparative studies attend to differences in 
means and variances while experimental studies attend 
only to differences in means. 

c. Cai:isal-comparative studies seek explanations on the 
basis of some characteristic of subjects, while 
experimental studies seek to determine effects of an 
additional Influence. 

d. The two are synoi^mous. 



16. All too often investigators state their null hypothesis 
merely as "the samples did not differ.” This leads to 
confusion because 

a. The samples are not adequately described. 

b. The dependent variable is undefined. 

c. The analysis technique is unstated. 

d. The basic assumptions of sampling are omitted. 

For each of the following research problems indicate which null 
hypothesis (I, II, or III) is most appropriate. 

17. An investigator wished to determine if right-handed or left- 
handed baseball pitchers revealed greater eccentricities as 
measured by Karl Olliver Obvious Key Induction Examination. 
He chose random sanq)les of 50 right-handed and left-handed 
pitchers from rosters of the teams in the American, 

National, and Pacific Coast leagues. 



18. After selecting three random samples of sixth grade students, 
an investigator gave one group intensive training in problem 
solving, a second group a similar amount of training in logic, 

-i' ■ no trcir.in- to a third ?rour. Ho then tested the cre-tive 
■*hilltiGs of the thr:e ''roups. 



19. An investigator wished to determine whether school admini- 
strators were more effective after completing his course. 

He watched his class members with other administrators in 
his state on the basis of sex, age, years of administrative 
experience, and number of hours of graduate credit. 



20. An investigator wished to varify his observation that boys 
were more quantitatively inclined than girls. He selected 
random samples of 10th grade boys and girls enrolled in 
his high school. 



Data Analysis II 
Criterion Examination 



For each of the following research situations describe the 
appropriate analysis technique. 

1. The Investigator wished to determine whether or not achieve- 
ment in arithmetic, as measured by a standardized test 
reporting scores in grade placement equivalents was related 
to nonverbal I.Q. 



2. An investigator asked the question "is college attendance 
predicted by high school grade point average, size of 
graduating class, and SAT verbal scores?" 



3. 



A group of judges observed teaching behavior exhibited by 25 
student teachers and ranked them according to their effec- 
tiveness. The investigators wished to determine if these 

rankings were related to average grades in professional 
courses received by the subjects. 
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4. An investigator determined the validity of his test by 

shoving that persons who scored higher on his test conpleted 
their degree programs while those who scored lower did not 
complete their degrees. This relationship could best be 
described by: 



5. To evaluate the effectiveness of a set of instructional 
materials, the researcher administered pre- and post- 
training tests (25-iten, multiple-choice test) to a 
group of college freshman English students. 



D. A coXltsgt; tfcuQent personnex ozxxCcjl wished to IuGW whether 
purchase of activities tickets differed among students 
living in dormitories, living in off-campus apartments 
or commuting from home. 



Reaction times were recorded for three types of students, at 
four different times of the day, under two ten^erature 
conditions for five consecutive days* 



A college political science professor was Interested in 
determining whether men or women tended to participate more 
frequently in anti-Viet Nam demonstrations held on campuses. 

All students enrolled in political science courses were 
polled. 



A biology Instructor teaching two sections of bacteriology 
required weekly lab attendance for members of one class and 
compared their performance on his final exam with perfonnance 
of the second class which experienced no lab sessions. 






10 . 



A salas imnager was Interested in determining which of 
several characteristics (annual inco!ae, age, size of 
family, etc.) were most accurate in differentiating 
Cadillac buyers from Corvair buyers. 



11. A dean of women postulated that women enrolled in the College 
of Education were more attractive than women enrulleu in the 
College of Arts and Sciences. Over a five year period she 
recorded the college affiliation of all women who were 
finalists in the 10 separate beauty contests held each year. 



12. A curious physiological psychology major wished to determine 
whether scholastic aptitude (SAT verbal score) was related 
to sex (male and female) . 





A college golf coach wished to determine whether there was a 
relationship between the order of finisli in his conference 
tournament and season average scores of nis team members. 



Ten pairs of Identical twins we’"e identified by a psycholo- 
gist. At random one twin from each pair was assigned to 
attend nursery school for one term. At the end of the term 
the psychologist obtained measures of social adaptivity for 
all 20 twins. Scores on the social adaptivity scale were 
such that they could be ranked in order of absolute 
magnitude. 



An investigator wishes to determine whether the 20 items on 
his test differ in difficulty. He administers the test to 
75 subjects and records pass-fail information for each itme 
for each subject. 



16 . 



An Investigcitor wished to detercine whether there was any 
association bett^een social class status and choice of high 
school cjrriculum. In a large high school he recorded the 
nunher of students in each of five social classes which 
chose each cf three curricula. 
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Evaluation Sheet 



We need to know how well the Ideas and issues In this manual are communicated 
to you. You are the test audience for this material. To remove or strengthen 
the weak spots, to retain or improve the strong ones (if any), an account of 
your learning experience ^ you read these sections is crucial. Use this sheet 
as you study and jot down your reactions. Don't be concerned with typo- 
graphical errors. We're concerned with how the message is coming through. 



SECTION # TITLE 



Give topic, 
paragraph or 
sentence, page 



Importance 

Rate 

1 2 3 4 5'" 
Low - High 



Clarity Suggested 

Rate Improvements 

1 2 3 4 5 
Low - High 



Over-all Rating of the Section: Use a 5-point Scale 



Important 

1 2 3 4 5 

Very Very 

Low High 

(Circle one) 



Understandable 
1 2 3 4 5 

Very Very 

Low High 

(Circle one) 



PLEASE PUT ANY ADDITIONAL COMMENTS ON BACK 
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